The study of the nervous tissue has proven very fecund in providing ideas to solve information processing problems. These solutions have been applied to practical problems such as data clustering, self organization or adaptive learning.
Artificial neural networks are computational simulations of biological neural networks such as found in the human brain. A typical artificial neural network includes set of nodes (i.e., models of neurons) whose inputs and outputs are connected to each other by links (i.e., models of synapses) to form a network. Each node processes its input signals to generate its output signals. Adaptive neural networks permit adaptation of the processing, allowing the neural network to learn.
The most common neural networks are idealized models that represent only the simplest features of a biological neural network. These models allow computationally efficient simulation of networks containing a large number of neurons, but they fail to represent subtler features of a biological neural network and have limited processing abilities. For example, in the McCulloch-Pitts neuron model the connection between two neurons is represented by a single scalar known as synaptic weight and the neuron is a linear operator composed of a non-linear threshold filter. Although this simplified neuron model works well when applied to problems like pattern recognition, because it does not represent the integration in time of incoming information it does not perform well in applications such as the identification of images in motion or the recognition of speech.
At the opposite end of the spectrum are artificial neural networks that use biophysically realistic models that closely resemble the biological structures and mechanisms, including accurate models of the detailed biophysics of neurons and synapses. Although these simulations based on such biophysical models provide more sophisticated processing capabilities, they have a much higher computationally cost. Consequently, biophysically realistic cortical simulations are currently limited in practice to networks having a relatively small number of nodes. In particular, this approach is far from being able to efficiently simulate a neural network with a size comparable with that of a human brain, which is estimated to have in excess of 1014 neurons. Moreover, because some computational properties of a neural network emerge only when the network is sufficiently large, the limited size of these networks also limits their capabilities.
In short, existing neural network techniques are either limited in their processing ability due to their use of simplistic models or unable to implement very large networks due to their use of complex biophysically realistic models.